Towards Autonomous Robotic Assembly: Using Combined Visual and Tactile Sensing for Adaptive Task Execution

Robotic assembly tasks are typically implemented in static settings in which parts are kept at fixed locations by making use of part holders. Very few works deal with the problem of moving parts in industrial assembly applications. However, having autonomous robots that are able to execute assembly tasks in dynamic environments could lead to more flexible facilities with reduced implementation efforts for individual products. In this paper, we present a general approach towards autonomous robotic assembly that combines visual and intrinsic tactile sensing to continuously track parts within a single Bayesian framework. Based on this, it is possible to implement object-centric assembly skills that are guided by the estimated poses of the parts, including cases where occlusions block the vision system. In particular, we investigate the application of this approach for peg-in-hole assembly. A tilt-and-align strategy is implemented using a Cartesian impedance controller, and combined with an adaptive path executor. Experimental results with multiple part combinations are provided and analyzed in detail

Segmentation and Coverage Planning of Freeform Geometries for Robotic Surface Finishing

Surface finishing such as grinding or polishing is a time-consuming task, involves health risks for humans and is still largely performed by hand. Due to the high curvatures of complex geometries, different areas of the surface cannot be optimally reached by a simple strategy using a tool with a relatively large and flat finishing disk. In this paper, a planning method is presented that uses a variable contact point on the finishing disk as an additional degree of freedom. Different strategies for covering the workpiece surface are used to optimize the surface finishing process and ensure the coverage of concave areas. Therefore, an automatic segmentation method is developed to find areas with a uniform machining strategy based on the exact tool and workpiece geometry. Further, a method for planning coverage paths is presented, in which the contact area is modeled to realize an adaptive spacing between path lines. The approach was evaluated in simulation and practical experiments on the DLR SARA robot. The results show high coverage for complex freeform geometry and that adaptive spacing can optimize the overall process by reducing uncovered gaps and overlaps between coverage lines

Unifying Skill-Based Programming and Programming by Demonstration through Ontologies

Smart manufacturing requires easily reconfigurable robotic systems to increase the flexibility in presence of market uncertainties by reducing the set-up times for new tasks. One enabler of fast reconfigurability is given by intuitive robot programming methods. On the one hand, offline skill-based programming (OSP) allows the definition of new tasks by sequencing pre-defined, parameterizable building blocks termed as skills in a graphical user interface. On the other hand, programming by demonstration (PbD) is a well known technique that uses kinesthetic teaching for intuitive robot programming, where this work presents an approach to automatically recognize skills from the human demonstration and parameterize them using the recorded data. The approach further unifies both programming modes of OSP and PbD with the help of an ontological knowledge base and empowers the end user to choose the preferred mode for each phase of the task. In the experiments, we evaluate two scenarios with different sequences of programming modes being selected by the user to define a task. In each scenario, skills are recognized by a data-driven classifier and automatically parameterized from the recorded data. The fully defined tasks consist of both manually added and automatically recognized skills and are executed in the context of a realistic industrial assembly environment

Autonomous Robot Planning System for In-Space Assembly of Reconfigurable Structures

Large-scale space structures, such as telescopes or spacecrafts, require suitable in-situ assembly technologies in order to overcome the limitations on payload size and mass of current launch vehicles. In many application scenarios, manual assembly by astronauts is either highly cost-inefficient or not feasible at all due to orbital constraints. However, (semi-) autonomous robotic assembly systems may provide the means to construct larger structures in space in the near future. Modularity is a key concept for such structures, and also for reducing costs in novel spacecraft designs. The advantage of the modular approach lies in the capability to generate a high number of unique assets from a reduced number of building blocks. Thus, spacecrafts can be easily adapted to particular use cases, and could even be reconfigured during their lifetime using a robotic manipulation system. These ideas lie at the core of our current EU project MOSAR (MOdular Spacecraft Assembly and Reconfiguration). Teleoperating a space robotic system from Earth to assemble $a$ modular structure is not straightforward. Major difficulties are related to time delays, communication losses, limited control modalities, and low immersion for the operator. Autonomous robotic operations are then preferred, and with this goal we propose $a$ fully autonomous system for planning in-space assembly tasks. Our system is able to generate assembly and reconfiguration plans for modular structures in terms of high-level actions that can autonomously be executed by $a$ robot. Through multiple simulation layers, the system automatically verifies the feasibility and correctness of action sequences created by the planner. The layers implement different levels of abstraction, hierarchically stacked to detect infeasible transitions and initiate replanning at an early stage. Levels of abstraction increase in complexity, ranging from a basic geometric description of the spacecraft, over kinematics of the robotic setup, to full representations of the actions. The system reuses information from failed checks in all layers to avoid similar situations during replanning. We use a hybrid approach where symbolic reasoning is combined with considerations of physical constraints to generate a holistic sequence of actions. We demonstrate our planner for large space structures in a simulation environment. In particular, we consider the reconfiguration of a given modular structure, i.e. disassemble parts and reassemble them in a new configuration. The adaptability of our planning system is shown by executing the assembly plans on robots with different sets of skills and in scenarios with simulated hardware failures

PULSAR: Testing the Technologies for On-Orbit Assembly of a Large Telescope

The EU project PULSAR (Prototype of an Ultra Large Structure Assembly Robot) carried out a feasibility analysis for a potential mission that could demonstrate robotic technology for autonomous assembly of a large space telescope. The project performed the analysis using two hardware demonstrators, one devoted to show the assembly of five segmented mirror tiles using a robotic manipulator, and another one showing extended mobility for assembling a large structure in low gravity conditions. The hardware demonstrators were complemented with a simulation analysis to demonstrate the operation of a fully integrated system and to address the challenges especially in the field of attitude and orbital control. The techniques developed in the project support the path toward In-Space Servicing, Assembly and Manufacturing (ISAM)

Schätzung des Kontaktzustands bei robotergestützten Montageaufgaben

Eine Herausforderung in der robotergestützten Montage mit dem KUKA Leichtbauroboter liegt in der automatischen Beurteilung, ob ein Montageprozess erfolgreich abgelaufen ist. Im hier entwickelten Schätzverfahren wird überprüft, ob die gemessenen Drehmomente und Geschwindigkeiten mit dem angenommenen Kontaktzustand konsistent sind. Alle geometrischen Unsicherheiten in der kinematischen Kette zwischen Roboter und Umgebung werden in einem Starrkörpermodell zu einer einzigen relativen Unsicherheit zusammengefasst, die mit einer sequentiellen Monte-Carlo Methode in der Form des Bootstrap-Filters geschätzt wird. Die kinematischen Einschränkungen im Kontakt werden durch virtuelle Kontaktmanipulatoren repräsentiert. Mehrfachkontakte werden mit Methoden der Grassmann-Cayley Algebra beschrieben. Eine Validierung erfolgt mit Hilfe von analytischen Kontaktmodellen für das klassische Peg-In-Hole Problem. Mehrere Experimente wurden hierfür mit dem Leichtbauroboter durchgeführt und offline ausgewertet

Adaptive Task Execution for Flexible Robotic Assembly Systems Using Intrinsic Tactile Sensing

Today's production requires flexible assembly systems that are able to manufacture individual products with little effort and can deal with uncertainties. In this thesis, a framework for autonomous robotic assembly is developed, which is based on reusable object-centric assembly skills. Using intrinsic tactile sensing in combination with the compliant control of light-weight robots, these skills can compensate for uncertainties and actively adapt to the present situation during task execution

Constraint-based Sample Propagation for Improved State Estimation in Robotic Assembly

In fast changing assembly scenarios, it is required to adapt the task execution to the current state of the setup without extensive calibration routines. Therefore, it is important to estimate the geometric uncertainties and contact states during the assembly execution. We use a sequential Monte Carlo (SMC) method to track the relative poses between workpieces during a robotic assembly based on joint torque and position measurements only. In contrast to existing approaches, we focus on assembly tasks where the workpiece is not fixed in the workcell, but can, for example, slide on a table surface. We propose a new constraint-based propagation model for the SMC approach: a compensation motion for the samples dependent on the violation of contact constraints is derived. This allows us to track the motion of the workpieces in cases where a common random diffusion model fails. The method is evaluated with experiments using an assembly scenario with two KUKA LBR iiwa robot arms and shows accurate tracking performance

Robotic Technologies for In-Space Assembly Operations

On-orbit robotic assembly is a key technology that can increase the size and reduce costs of construction of large structures in space. This document provides an overview of existing or emerging robotic technologies for space-born assembly, including also the development of standard interfaces for connectivity that combine mechanical connections with electronic and power signals. Technologies that can enable on-orbit assembly demonstrations in the near future are currently under development at the Institute of Robotics and Mechatronics in the German Aerospace Center - DLR, as showcased in a setup for autonomous assembly of structures made out of standard aluminium profiles

Robust, Locally Guided Peg-in-Hole using Impedance-Controlled Robots

We present an approach for the autonomous, robust execution of peg-in-hole assembly tasks. We build on a sampling-based state estimation framework, in which samples are weighted according to their consistency with the position and joint torque measurements. The key idea is to reuse these samples in a motion generation step, where they are assigned a second task-specific weight. The algorithm thereby guides the peg towards the goal along the configuration space. An advantage of the approach is that the user only needs to provide: the geometry of the objects as mesh data, as well as a rough estimate of the object poses in the workspace, and a desired goal state. Another advantage is that the local, online nature of our algorithm leads to robust behavior under uncertainty. The approach is validated in the case of our robotic setup and under varying uncertainties for the classical peg-in-hole problem subject to two different geometries